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ENHANCING AUDIO SIGNALS BY NONLINEAR SPECTRAL 

OPERATIONS 

CROSS REFERENCE TO RELATED APPLICATIONS 

This application is related to co-pending U.S. Patent Application No. 

5 (Attorney Docket No. CLABP203) entitled TRANSIENT DETECTION 

AND MODIFICATION IN AUDIO SIGNALS, filed concurrently herewith, which is 
incorporated herein by reference for all purposes. 

FIELD OF THE INVENTION 

The present invention relates generally to digital signal processing. More 
10 specifically, enhancing audio signals by nonlinear spectral operations is disclosed. 

BACKGROUND OF THE INVENTION 

An audio signal may comprise one or more regions of relatively rapid change in 
spectral content, such as without limitation a transient audio event. Such regions of 
relatively rapid change may occur across a broad spectrum or within a more narrow range 
15 of frequencies. The perceptual experience of one listening to an audio signal may be 
affected by the manner in which such regions of the signal are rendered. For example, a 
listener may perceive an audio signal in which such rapid changes in spectral content are 
emphasized or enhanced as being more "edgy" or "crisp". By contrast, one may perceive 
an audio signal in which such rapid changes in spectral content have been smoothed or 
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otherwise de-emphasized as being more "warm" or "smooth". Different listeners may 
have different preferences regarding how "edgy" or "smooth" a particular audio signal is 
rendered. 

An analogy can be made to a visual image. In a visual image, the feature 
5 analogous to a region of rapid change in spectral content is a line defining a boundary 
between a region of one color and a region of a second, contrasting color. Visual image 
processing techniques are known to "soften" or "blur", or conversely to "sharpen", such 
transitions, as desired, in order to enhance the perceptual experience of a viewer in a 
desired manner. 

1 0 Typical prior art static equalization techniques are not fully satisfactory for this 

purpose, as they do not respond dynamically to the audio signal being processed and may 
not as a result provide the desired listening experience for a given signal 

Therefore, there is a need for a way to process an audio signal to either smooth or 
enhance rapid changes in spectral content to provide a desired perceptual experience with 
15 respect to such changes. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention will be readily understood by the following detailed 
description in conjunction with the accompanying drawings, wherein like reference 
numerals designate like structural elements, and in which: 

5 Figure 1 is a flowchart illustrating a process used in one embodiment to enhance 

audio signals by nonlinear spectral operations. 

Figure 2 is a flowchart illustrating a process used in one embodiment to apply a 
nonlinear modification to the STFT results for successive portions of a time-domain 
signal a(t), as in step 106 of the process shown in Figure 1. 

10 Figure 3 is a block diagram illustrating a system used in one embodiment to apply 

a nonlinear modification to the STFT results A(co,n) for successive frames V of a time- 
domain audio signal a(t), as in step 106 of the process shown in Figure 1. 

Figure 4 illustrates the response of a linear filter such as may be used in one 
embodiment to determine a modified spectral magnitude for a portion of an audio signal, 
15 as in the filter block 306 of Figure 3. 

Figure 5 is a filter response graph illustrating the characteristics of a high-shelf 
IIR filter, such as may be used in one embodiment to enhance the high modulation 
frequency portions of an audio signal. 
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Figure 6 illustrates the filter response of a filter used in one embodiment to 
smooth or otherwise de-emphasize portions of an audio signal having a high modulation 
frequency. 

Figure 7 illustrates a user control provided in one embodiment to enable a user to 
indicate the degree of sharpening or smoothing desired for a particular audio signal. 

Figure 8 is an illustration of a set of controls 800 in which separate controls are 
provided for setting the cut-off modulation frequency and the exponent. 

Figure 9 is a flowchart illustrating a process used in one embodiment to provide 
modification selectively in two or more frequency bands. 

Figure 10 illustrates a set of boundary frequency controls 1000 provided in one 
embodiment to enable a user to control the lower and upper limits, respectively, of the 
frequency bands in an embodiment in which two frequency bands may be defined. 
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DETAILED DESCRIPTION 

It should be appreciated that the present invention can be implemented in 
numerous ways, including as a process, an apparatus, a system, or a computer readable 
medium such as a computer readable storage medium or a computer network wherein 
5 program instructions are sent over optical or electronic communication links. It should 
be noted that the order of the steps of disclosed processes may be altered within the scope 
of the invention. 

A detailed description of one or more preferred embodiments of the invention is 
provided below along with accompanying figures that illustrate by way of example the 

10 principles of the invention. While the invention is described in connection with such 
embodiments, it should be understood that the invention is not limited to any 
embodiment. On the contrary, the scope of the invention is limited only by the appended 
claims and the invention encompasses numerous alternatives, modifications and 
equivalents. For the purpose of example, numerous specific details are set forth in the 

1 5 following description in order to provide a thorough understanding of the present 

invention. The present invention may be practiced according to the claims without some 
or all of these specific details. For the purpose of clarity, technical material that is known 
in the technical fields related to the invention has not been described in detail so that the 
present invention is not unnecessarily obscured. 

20 Enhancing audio signals by nonlinear spectral operations is disclosed. In one 

embodiment, regions of rapid change of spectral content are either enhanced or 
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smoothed, as desired, by applying a nonlinear modification to the spectral magnitude of 
the Short Time Fourier Transform (STFT) for the region. The STFT of a time-domain 
signal represents the spectral content in the frequency domain of a series of successive 
frames of a time-domain audio signal, the frame being defined by the length (in time) of a 
5 "window" which extracts a portion of the time-domain signal for frequency-domain 
analysis. By performing the STFT operation on a chronological succession of frames, 
one can analyze how the spectral magnitude of a signal changes over time. 

As described above, it may be desirable to enhance or smooth rapid changes in 
spectral content, in order to provide a desired perceptual experience, sometimes referred 

10 to as the "texture" of the audio signal as rendered, to a listener. Regions of rapid spectral 
change are associated with a high modulation frequency in affected frequency bands. 
That is, for portions of an audio signal characterized by a rapid rate of change in spectral 
content, as determined by comparing the STFT results for successive frames, the 
modulation frequency is higher than for regions of less dramatic change in spectral 

15 content. As described more fully below, in one embodiment nonlinear spectral 

operations are performed to enhance high modulation frequency portions of an audio 
signal. 

Figure 1 is a flowchart illustrating a process used in one embodiment to enhance 
audio signals by nonlinear spectral operations. The process begins in step 102 in which a 
20 time-domain audio signal a(t) is received. In step 104, the short time Fourier transform 
(STFT) is calculated for successive (and in one embodiment overlapping) portions of the 
time-domain signal a(t). In step 106, a nonlinear modification is applied to the STFT 
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result for each successive portion of the time-domain audio signal a(t). In step 108, the 
inverse STFT is performed for the successive modified STFT results to produce a 
modified time-domain signal a'(t). In one embodiment, step 108 comprises performing 
an overlap-add operation of the inverse STFT results to make a complete time-domain 
5 signal. While the process shown in Figure 1 specifies by way of example performing the 
STFT and/or inverse STFT, other subband filter blocks may be used in other 
embodiments. 

Figure 2 is a flowchart illustrating a process used in one embodiment to apply a 
nonlinear modification to the STFT results for successive portions of a time-domain 

10 signal a(t), as in step 106 of the process shown in Figure 1 . In step 202, the STFT result 
A(co,n) for successive frames "n" of the time-domain signal a(t) is received. In one 
embodiment, the successive frames "n" may overlap to a degree in the time domain. In 
one embodiment, as each successive STFT result is calculated, the result is received in 
step 202, which repeats as each successive result is received. In step 204, the spectral 

15 magnitude S(co,n) is calculated for each successive STFT result A(a),n) as it is received in 
step 202. In step 206, the spectral magnitude values S(co,n) are processed with a 
nonlinear filter to determine modified spectral magnitudes S f (oo,n). In one embodiment, 
the nonlinear filter comprises one or more filters configured to amplify or otherwise 
enhance high modulation frequency portions of the audio signal. In one embodiment, the 

20 nonlinear filter comprises one or more filters configured to smooth or otherwise de- 
emphasize high modulation frequency portions of the audio signal. In step 208 a 
modification ratio is calculated by dividing the modified magnitudes S'(co,n) by the 
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corresponding original magnitudes S(co,n). In step 210, a nonlinear expansion or 
compression, as desired, is applied to the modification ratio calculated in step 208 to 
determine a final modification factor. In step 212, the final modification factor 
determined in step 210 is applied to the original STFT results received in step 202 to 
5 yield a modified STFT result for each successive portion of the audio signal. 

Figure 3 is a block diagram illustrating a system used in one embodiment to apply 
a nonlinear modification to the STFT results A(co,n) for successive frames "n" of a time- 
domain audio signal a(t), as in step 106 of the process shown in Figure 1 . The system 
300 receives the STFT result A(co,n) for successive frames "n" of the time-domain audio 

10 signal a(t) on line 302. The STFT results are provided to a magnitude determination 
block 304, which is configured to calculate the spectral magnitude S(oo,n) for each 
successive frame "n". The spectral magnitude values S(co,n) are provided to a filter block 
306. In one embodiment, the filter block 306 is configured to either enhance or de- 
emphasize, as desired, portions of the audio signal having a high modulation frequency 

1 5 relative to other portions. 

The "modulation frequency" of an audio signal (or portion thereof) describes the 
frequency content of the time evolution of a portion of the signal spectrum such as a 
designated frequency bin. In one embodiment, the spectral magnitudes S(co,n) for 
successive windows 4 'n" are processed to extract the portion of each successive result that 
20 pertains to a frequency bin of interest, the frequency bin of interest being defined by an 
upper and lower cut-off frequency. In one such embodiment, the portions of the spectral 
magnitudes S(co,n) relevant to a frequency bin coo for successive frames "n" comprise a 
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time- varying spectral magnitude value S(coo,t) representing how the spectral content of 
the underlying audio signal changes over time within the frequency bin. In one 
embodiment, in order to identify portions of the audio signal for which the spectral 
content is changing rapidly, the Fourier transform is performed on the spectral magnitude 
5 values S(coo,t), the result of which may be represented graphically as a plot of the Fourier 
transform magnitude on the vertical axis and the modulation frequency, i.e., the rate at 
which the spectral content changes in the applicable frequency bin, on the horizontal axis, 
with a high value for the Fourier transform magnitude indicating a significant degree of 
change within the bin at the corresponding modulation frequency. In one embodiment, 
10 filter block 306 is configured to enhance (or smooth, as desired) high modulation 
frequency components of the audio signal (within the signal bin of interest, in one 
embodiment) relative to low modulation frequency components, with the result that 
portions of the audio signal that are associated with rapid changes in spectral content 
(e.g., within the frequency bin) are enhanced (or smoothed). 

15 In one embodiment, the filter block 306 comprises a linear filter. In one 

embodiment, the filter block 306 comprises a nonlinear filter. In one embodiment, the 
filter block 306 comprises an infinite impulse response (IIR) filter. In one embodiment, 
the filter block 306 comprises a plurality of linear (and/or nonlinear) filters and the 
system is configured to switch between the filters based on one or more signal-dependent 

20 parameters, as described more fully below in connection with Equations [2] and [5] 

below. The filter block 306 is configured to provide to a modification ratio determination 
block 312 via line 308 a series of modified spectral magnitude values S'(co,n). The 
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original unmodified spectral magnitude values S(co,n) generated by magnitude 
determination block 304 are also provided via line 310 to modification ratio 
determination block 312. The modification ratio determination block 3 12 is configured 
in one embodiment to divide the modified spectral magnitude values S'(a>,n) by the 
5 corresponding original, unmodified spectral magnitude values S(a),n) to calculate a 
modification ratio equal to S'(co,n)/S(oo,n). The modification ratio S'(cG,n)/S(to,n) is 
provided to a nonlinear expansion/compression block 314. The nonlinear 
expansion/compression block 314 is configured in one embodiment to raise the 
modification ratio to an exponent a. Whereas the filter block 306 determines the nature 

10 of the modification, e.g. smoothing or sharpening, of high modulation-frequency signal 
components, the extent of the smoothing or sharpening is determined both by the filter 
block 306 and also by the nonlinear compression/expansion block 314 by means of the 
exponent a. In the embodiment shown in Figure 3, when a=0, the output of the nonlinear 
compression/expansion block 314 is equal to 1 and no modification occurs. For a=l, the 

1 5 nonlinear compression/expansion block 3 14 is equal to the ratio SXco,n)/S(co,n) and the 
magnitude of the output on line 322 is that of the filter output 308. For a increasing from 
0 to 1, the extent of the modification gradually increases from the former case to the latter 
case. For a > 1, the modification effect of the filter block 306 is further amplified; in 
other words, the effect of either the smoothing or sharpening is further increased. In one 

20 embodiment, the value of the exponent a is determined at least in part by a user input as 
discussed more fully below. 
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The nonlinear compression/expansion block 314 provides as output on line 316 a 
series of modification function values M(a>,n) equal to the modification ratios received by 
block 314 raised to the exponent a, as indicated by the following equation [1]: 



[1] M(a,n)= —f f 

S(co,n) 



5 In one alternative embodiment, the nonlinear compression/expansion block 314 is 

configured to raise only the numerator portion of the modification ratio to the exponent a, 
which in one such embodiment increases the effect of the modification across 
frequencies. In one such embodiment, the nonlinear compression/expansion block 3 14 is 
further configured to raise the entire result [S , (co,n)]7S(co,n) to a second exponent y, 

10 which controls the amount of the modification. Those of skill in the art will recognize 
that nonlinear compression/expansion block 314 could be configured to perform 
numerous different types of nonlinear modification, and any set of processes and/or 
components configured to perform any such nonlinear modification may be used to 
implement the nonlinear compression/expansion block 314, and any such implementation 

15 is within the scope of the present disclosure. 

Referring further to Figure 3, the modification function values provided on line 
316 are provided as a bias circuit input to variable gain amplifier 318. The variable gain 
amplifier 318 receives as input the original, unmodified STFT results received by the 
system 300 via line 302 and provided via line 320 to the variable gain amplifier 318. The 
20 variable gain amplifier 318 provides as output on line 322 a modified STFT result, which 
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result has the same phase as the original STFT result received via lines 302 and 320 with 
the magnitude modified in accordance with the modification function received via line 
316. 

In one embodiment, the system 300 of Figure 3 may be implemented by 
5 programming a processor to perform the above-described operations associated with each 
functional block illustrated in Figure 3. In one embodiment, the processor may comprise 
a general use processor, such as may be found in a typical personal computer. In one 
embodiment, the processor may comprise a processor specially configured to process 
audio data, such as a processor comprising part of a sound card and/or an embedded 
10 digital signal processor (DSP). In one embodiment, a special purpose computer chip, 
such as an ASIC, may be configured to implement the system illustrated in Figure 3. In 
one embodiment, one or more circuits or other hardware may be used to implement one 
or more portions of the system shown in Figure 3, such as without limitation filter block 
306. 

15 Figure 4 illustrates the response of a linear filter such as may be used in one 

embodiment to determine a modified spectral magnitude for a portion of an audio signal, 
as in the filter block 306 of Figure 3. In one embodiment, the filter block 306 of Figure 3 
may comprise one or more linear filters such as the one illustrated in Figure 4, each 
having its own characteristics and response curve similar to the one shown in Figure 4, 

20 with the system being configured to provide a nonlinear response by switching between 
the linear filters based on one or more signal-dependent parameters. As the response 
curve shown in Figure 4 indicates, the associated filter would provide relatively little 
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modification to low modulation frequency portions of the audio signal, and much greater 
modification to higher modulation frequency portions of the audio signal. In one 
embodiment, a filter having characteristics similar to those illustrated in Figure 4 may 
comprise a portion of the filter block 306 of the system shown in Figure 3, in an 
5 embodiment in which the filter block 306 may be used to enhance portions of the audio 
signal characterized by a high modulation frequency. 

Figure 5 is a filter response graph illustrating the characteristics of a high-shelf 
IIR filter, such as may be used in one embodiment to enhance the high modulation 
frequency portions of an audio signal. As shown in Figure 5, a high-shelf filter provides 
10 no modification to the portions of a signal characterized by a low modulation frequency 
and a relatively high degree of modification to portions characterized by a high 
modulation frequency. In one embodiment, a high-shelf IIR filter such as the one 
illustrated in Figure 5 may be included in the filter block 306 of the system shown in 
Figure 3. 

15 Figure 6 illustrates the filter response of a filter used in one embodiment to 

smooth or otherwise de-emphasize portions of an audio signal having a high modulation 
frequency. In one embodiment, the filter illustrated in Figure 6 comprises an IIR low- 
shelf filter. As shown in Figure 6, the IIR low-shelf filter provides no modification to 
portions of the audio signal characterized by a low modulation frequency and a relatively 

20 high degree of suppression of portions of the audio signal characterized by a high 
modulation frequency. In one embodiment, one or more of the filters illustrated in 
Figures 4, 5, and 6, and other filters as required in a particular application, may be 
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included in the filter block 306 of the system shown in Figure 3, and invoked as 



appropriate to modify the audio signal being processed by the system 300 shown in 



Figure 3, depending on whether portions of the audio signal characterized by a high 



modulation frequency are to be enhanced or de-emphasized, as desired. 



5 



In one embodiment, depending on the operation being performed by the filter 



block 306, one or more characteristics of one or more filters used to process the spectral 
magnitudes S(co,n) may be determined by a user input. In one embodiment, the user 
input comprises a cut-off modulation frequency which is mapped to two time constant 
coefficients X\ and A, 2 , which in one embodiment have different values in order to provide 
10 the characteristics of a nonlinear IIR filter. In one embodiment, where portions of the 
audio signal characterized by a high modulation frequency are to be de-emphasized or 
smoothed, the nonlinear filter applied by the filter block 306 to the successive magnitude 
values S(co,n) of the STFT results received by the system 300 is characterized by the 
following equation [2]: 



In one embodiment, for portions of an audio signal in which high modulation 
frequency regions are to be enhanced, a high-pass finite impulse response (FIR) filter (i.e. 
a sharpener) is applied to the STFT magnitudes, in accordance with the following 
equation [3]: 



15 



[2] S'(o>,n) = 



X x $ (G) 9 n - 1) + (1 - \ )S(a), n) if S(co 9 n) < V (co, n-1) 
X 2 S' (a), n - 1) + (1 - X 2 )S{co, n) if S(a), n) > S' (a, n - 1) 



20 



[3] 



S' (a), n) = g[(fi - 1)5(10, »-!) + (£ + \)S(co, n)] 
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where the gain g is computed in accordance with the following equation [4]: 



[4] g = — 

1 J * 2J3 



and p is in one embodiment a fixed parameter. In one alternative embodiment, p varies 
based on one or more signal-dependent parameters. In one embodiment, the value of p is 
5 within the range 0 < p < 1 . 

In one embodiment, when the STFT magnitudes are enhanced in accordance with 
the above equation [3], the output of the high-pass FIR filter is smoothed using an IIR 
smoother to reduce artifacts. In one embodiment, the behavior of the smoother is similar 
to the IIR low-pass filter described above in equation [2] and is therefore characterized by 
10 the following equation [5]: 



[5] S'(a>,n) = 



\ V (<y , n - 1) + (1 - A 3 )S(co, n) if S(co, n) < S\co,n-l)] 
[Z 4 S* (<y, * - 1) + (1 - Z 4 )S(o), n) if S(<D, n) > S'(a>, n - 1)1 



In one embodiment, the respective value of the coefficients A, 3 and A, 4 in equation [5] may 
be determined at least in part by a user input. In one alternative embodiment, the values 
of the coefficients X 3 and A, 4 are fixed and are determined in advance by a sound designer. 

15 Figure 7 illustrates a user control provided in one embodiment to enable a user to 

indicate the degree of sharpening or smoothing desired for a particular audio signal. The 
control 700 comprises a level indicator 702 which may be positioned along a slider 704 
to indicate the degree of sharpening and/or smoothing desired. While a slider type 
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control is shown in Figure 7, any type of control may be used, including without 
limitation a knob, rocker switch, or any other suitable control. The slider 704 has a null 
point 706 corresponding in one embodiment to no modification of the audio signal, i.e. 
the modified spectral magnitudes S'(co,n) would be equal to the original magnitudes 
5 S(co,n). In one embodiment, such a result is achieved by not processing the spectral 
magnitudes with any nonlinear filter and/or by setting the values of filter coefficients as 
required to ensure that the output values S f (co,n) equal the original values S(co,n), such as 
by setting values X\ and X 2 to be equal to zero in equation [2]. The slider 704 has a 
maximum smoothing position 708 corresponding to a maximum degree of de-emphasis 

10 of high modulation frequency portions of the audio signal. In one embodiment, the 

maximum smoothing position 708 corresponds to maximum values for the coefficients Xi 
and X 2 of the filter corresponding to equation [2], and/or to a maximum value for the 
exponent a (see equation [1], e.g.). The slider 704 has a maximum sharpening position 
710 corresponding to a maximum degree of emphasis of high modulation frequency 

15 portions of the signal. In one embodiment the maximum sharpening position 710 

corresponds to a maximum value for the exponent a, a minimum value for the coefficient 
P in the high-pass filter of Equation [3], and minimum values for the time coefficients k 3 
and I4 in the artifact smoother of Equation [5]. 

In one embodiment, intermediate positions between the null position 706 and the 
20 maximum positions 708 and 710, respectively, are determined by employing a sound 
designer to determine one or more set points between the null and respective maximum 
positions. Such a sound designer may choose intermediate set point values for the 
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coefficients X\ and X 2 , the exponent a, and/or the sharpening/smoothing filter gain, as 
applicable, to achieve a pleasing listening experience at each set point between the null 
and respective maximum values, with set points nearer to the null position 706 in one 
embodiment being characterized by less modification of high modulation frequency 
5 portions of the audio signal than set points nearer to one or the other of the maximum 
positions 708 and 710. Once a sound designer has selected one or more set points 
between the null position 706 and the respective maximum positions 708 and 710, 
intermediate values for the coefficients X\ and Aa, the exponent a, and/or the 
sharpening/smoothing filter gain, as applicable, corresponding to positions between the 

10 set points or between a set point and the null position 706 and/or one of the maximum 
positions 708 and 710 respectively may be determined using known interpolation 
techniques. In one embodiment, the interpolation of the underlying values for the 
coefficients X\ and X, 2 , the exponent a, and/or the sharpening/smoothing filter gain, as 
applicable, corresponding to positions between set points may be either linear or 

1 5 nonlinear, as may be determined to be most appropriate given the set of set points 
designed by the sound designer. 

Figure 8 is an illustration of a set of controls 800 in which separate controls are 
provided for setting the cut-off modulation frequency and the exponent. The set of 
controls 800 comprises a cut-off modulation frequency slider control 802 and an 
20 exponent slider control 804. The cut-off modulation frequency slider control 802 

comprises a cut-off modulation frequency slider 806 and a cut-off modulation frequency 
level indicator 808 configured to be positioned along the slider 806 to indicate a cut-off 
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modulation frequency between a minimum value corresponding to a minimum cut-off 
modulation frequency position 812 and a maximum value corresponding to a maximum 
cut-off modulation frequency position 810. In one embodiment, for smoothing the 
maximum cut-off modulation frequency position 810 corresponds to a minimum value 
5 for the coefficients \\ and X 2 of equation [2], which in turn corresponds to a minimum 
level of modification to the audio signal. In one embodiment, the minimum cut-off 
modulation frequency position 812 corresponds to a maximum value for the coefficients 
X\ and X 2 in equation [2], which in turn corresponds to a maximum degree of 
modification of the audio signal. In one embodiment, for sharpening the control 802 is 
10 used to control the value of the coefficient p in equation [3], in an embodiment in which 
the value of p may be varied. 

The exponent slider control 804 comprises a slider 814 and an exponent value 
indicator 816 configured to be moved along the slider 814 to indicate a desired value for 
the exponent a. The control 804 comprises a minimum position 818 corresponding to a 
1 5 minimum value for the exponent a and a maximum position 820 corresponding to a 

maximum value for the exponent a. In one embodiment, the minimum value of a is a=0 
(i.e., no modification of the audio signal) and the maximum value is a maximum value 
greater than 0 as determined by a sound designer. 

The systems and methods described herein may in one embodiment be applied 
20 selectively to one or more frequency bands (or "bins") associated with the audio signal. 
In one embodiment, for example, two separate frequency bands may be identified for 
modification and modifications applied selectively to each frequency band, such that the 
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audio signal may be modified differently in one frequency band than in the other. In one 
embodiment, each frequency band subject to modification may be defined by a lower and 
upper boundary frequency. 

Figure 9 is a flowchart illustrating a process used in one embodiment to provide 
5 modification selectively in two or more frequency bands. In one embodiment, the 
process shown in Figure 9 may be used to apply a nonlinear modification to the STFT 
results for each successive portion of the time-domain audio signal a(t), as in step 106 of 
Figure 1 . That is, in one embodiment the process shown in Figure 9 may be used as a 
substitute to the process shown in Figure 2 in an embodiment in which the audio signal 
10 may be modified differently in two or more frequency bands of interest. 

The process shown in Figure 9 begins in step 902 in which the STFT results for 
successive portions of the time-domain signal a(t) are received. In step 904, the STFT 
result for each successive window is divided into the frequency bands of interest. In one 
embodiment, the STFT results are divided into frequency bands by extracting the values 

15 S(co,n) for frequencies within the band. In one alternative embodiment, the STFT results 
are divided into frequency bands by processing the STFT magnitudes S(co,n) using a band 
pass filter frequency response F,(o)), such that the spectral content Xi(co,n) within the i-th 
band is determined by the equation Xj(co,n) = Fi(co)S(oo,n). In step 906, the spectral 
magnitude of the STFT result for each successive portion of the time-domain audio signal 

20 a(t) is calculated for each frequency band of interest. In step 908, a modified spectral 
magnitude is calculated for each band. In step 910, the modified spectral magnitude 
calculated in step 908 for each band is used to calculate a modified STFT result for each 
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band. In one embodiment, step 910 comprises a process similar to that described above 
in connection with steps 208, 210 and 212 of the process shown in Figure 2. In step 912, 
the modified STFT results within each band are synthesized to form a combined modified 
STFT result for the audio signal In step 914, the combined modified STFT result 
5 synthesized in step 912 is provided as output. In one embodiment, the output provided in 
step 914 is then processed by applying the inverse STFT, as described above in 
connection with step 108 of the process shown in Figure 1, to provide a modified time- 
domain signal a'(t). 

Figure 10 illustrates a set of boundary frequency controls 1000 provided in one 
10 embodiment to enable a user to control the lower and upper limits, respectively, of the 
frequency bands in an embodiment in which two frequency bands may be defined. The 
set of controls 1000 comprises a lower boundary frequency slider 1002 associated with a 
lower frequency boundary indicator 1004 configured to be positioned along the slider 
1002 to indicate the lower boundary of the lower frequency band. The set of controls 
15 1000 further comprises a middle boundary frequency slider 1006 associated with a 

middle boundary frequency indicator 1008 configured to be positioned along the slider 
1006 to indicate the upper boundary frequency for the lower frequency band and the 
lower boundary frequency for the upper frequency band. In one alternative embodiment, 
the upper boundary of the lower frequency band may be different than the lower 
20 boundary of the upper frequency band, and in such an embodiment two separate controls 
are provided to allow a user to indicate the upper boundary of the lower frequency band 
and the lower boundary of the upper frequency band respectively. The set of controls 
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1000 further comprises an upper boundary frequency slider 1010 associated with an 
upper boundary frequency indicator 1012 configured to be moved along the slider 1010 
to indicate the upper boundary of the upper frequency band. In one embodiment, the 
audio signal is not modified at all with respect to frequencies below the lower boundary 
5 coi and above the upper boundary co 3 . In one embodiment, the boundary frequency 

controls 1000 may be used in connection with other controls, such as those illustrated in 
Figures 7 and 8, to enable a user to control both the definition of the frequency bands and 
the level of modification within each band. In one embodiment, a control such as the 
control 700 shown in Figure 7 is provided for each frequency band. In one embodiment, 
10 a set of controls such as the controls 800 of Figure 8 are provided for each frequency 
band. While two bands are described above, in other embodiments three or more bands 
may be used. 

Although the foregoing invention has been described in some detail for purposes 
of clarity of understanding, it will be apparent that certain changes and modifications may 
15 be practiced within the scope of the appended claims. It should be noted that there are 
many alternative ways of implementing both the process and apparatus of the present 
invention. Accordingly, the present embodiments are to be considered as illustrative and 
not restrictive, and the invention is not to be limited to the details given herein, but may 
be modified within the scope and equivalents of the appended claims. 

20 WHAT IS CLAIMED IS : 
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